Multimodal interaction on mobiles and applications∗

The current generation of mobile phones allows traditional interactions via menus and buttons, but offers a variety of new ways of interacting with the device via the built-in sensors. Multimodality through mobile phones are growing in importance and aims to create applications which are socially acceptable and enjoyable. We present multiple multimodal applications made for pedestrians and impaired people. We also talk about more common applications such as games, photos retrieval system and home entertainment systems. For each of them, we describe which modalities are used and how they are combined. The results are analysed in order to determine if the use of multimodality has benefits. Finally, we conclude that multimodality is very efficient in almost all cases and has a great potential in a near future.